A Theory of Self-organising Neural Networks

The purpose of this paper is to present a generalisation of the probabilistic approach to the static analysis of selforganising neural networks that appeared in [1]. In the simplest case the network has two layers: an input and an output layer. An input vector is used to clamp the pattern of activity of the nodes in the input layer, and the resulting pattern of individual ring events of the nodes in the output layer is described probabilistically. Finally, an attempt is made to reconstruct the pattern of activity in the input layer from knowledge of the location of the ring events in the output layer. This inversion from output to input is achieved by using Bayes' theorem to invert the probabilistic feed-forward mapping from input to output. A network objective function is then introduced in order to optimise the overall network performance. If the average Euclidean error between an input vector and its corresponding reconstruction is used as the objective function, then many standard self-organising networks emerge as special cases [1, 2]. In Section II the network objective function is introduced, in Section III a simpler form is derived which is an upper bound to the true objective function, and in Section IV the derivatives with respect to various parameters of this upper bound are derived. Finally, in Section V various standard neural networks are analysed within this framework.